Image synthesis device for electronic mirror and method thereof

ABSTRACT

According to one embodiment, an image synthesis device for an electronic mirror includes a rear camera which obtains a first image from a first position, and a side camera which obtains a second image of the same direction with a view of the rear camera from a second position. The second image includes a view obstruction. When a part of the first image is connected as a complementary image to the view obstruction of the second image, an image processing device converts an image of the view obstruction into a translucent image, superimposes the complementary image on the translucent image, and obtains a third image which remains an outline of the complementary image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit of priorityunder 35 U.S.C. § 120 from U.S. application Ser. No. 16/661,439 filedOct. 23, 2019, which is a continuation of U.S. application Ser. No.15/659,725 filed Jul. 26, 2017 (now U.S. Pat. No. 10,506,178 issued Dec.10, 2019), and claims the benefit of priority under 35 U.S.C. § 119 fromJapanese Patent Application No. 2017-008584 filed Jan. 20, 2017, theentire contents of each of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an image synthesisdevice for an electronic mirror and a method thereof.

BACKGROUND

For example, an electronic mirror is configured to cause a display todisplay the images obtained by a plurality of cameras provided in avehicle, instead of the images of the conventional rearview or sidemirrors.

In the system of the electronic mirror, the cameras are attached todifferent positions in the vehicle. Thus, when the cameras capture theview behind the vehicle in the same direction, a part area included inan effective region which can be captured by the field of view of one ofthe cameras may be a blind region (ineffective region) which cannot becaptured by the field of view of another camera.

As described above, in the electronic mirror, when the cameras capturethe view behind the vehicle in the same direction, an effective regionand an ineffective region differ among the cameras.

BRIEF DESCRIPTION OF THE DRAWINGS

A general architecture that implements the various features of theembodiments will now be described with reference to the drawings. Thedrawings and the associated descriptions are provided to illustrate theembodiments are not to limit the scope of the invention.

FIG. 1 is a plan view showing a vehicle (car).

FIG. 2 is a side view showing the vehicle of FIG. 1.

FIG. 3 is a block configuration diagram showing a system of anelectronic mirror to which one embodiment is applied.

FIG. 4 show images for explaining an example of the basic operation ofthe system of the electronic mirror of FIG. 3.

FIG. 5 show images for explaining an example of operation includingprojective transformation in the system of the electronic mirror of FIG.3.

FIG. 6 is an explanatory diagram shown for explaining the basicprinciple of projective transformation.

FIG. 7 is an explanatory diagram showing an example of a determinant forprojective transformation.

FIG. 8 is an explanatory diagram shown for explaining a process foraveraging the luminance of the image of a side of the vehicle.

FIG. 9 is an explanatory diagram showing another example for generatingan image superimposed on an absolute blind region.

FIG. 10 is an explanatory diagram showing an example in which a pseudoimage is superimposed on an absolute blind region.

FIG. 11 is an explanatory diagram showing another example in which apseudo image is superimposed on an absolute blind region.

FIG. 12 is an explanatory diagram showing an example in which thedistance from the vehicle moving on a road to a reference plan position(a far plan position or a ground position) is changed in accordance withthe speed.

FIG. 13 shows an example of the image of the electronic mirror when thedistance from the vehicle to the reference (virtual) plan position isincreased to the distance to a far plan position.

FIG. 14 shows an example of the image of the electronic mirror when thedistance from the vehicle to the reference (virtual) plan position isdecreased to the distance to a close plan position.

DETAILED DESCRIPTION

Various embodiments will be described hereinafter with reference to theaccompanying drawings.

The embodiments provide an image synthesis device for an electronicmirror and a method thereof in which, even when a field of view includesan obstruction, the image of the obstruction is made translucent and isreplaced by an effective image (complementary image).

In general, according to one embodiment, an image synthesis device foran electronic mirror comprises:

a first camera which obtains a first image of a first view by a firstfield of view from a first position;

a second camera which obtains a second image by a second field of viewfrom a second position different from the first position, the secondfield of view including a view obstruction in a direction of the firstview; and

an image processing device which converts, when a part of the firstimage is connected as a complementary image to a part of the secondimage, an image of the view obstruction included in the second imageinto a translucent image, superimposes the complementary image on thetranslucent image, and obtains a third image by providing a borderbetween the first image and the complementary image in an outline of thetranslucent image and connecting the first image and the complementaryimage.

An embodiment will further be described with reference to the drawings.

FIG. 1 is a plan view showing a vehicle 100. FIG. 2 is a side view ofthe vehicle shown in FIG. 1.

The vehicle 100 comprises a first camera 101 on a rear trunk or near alicense plate. The first camera 101 is capable of capturing the rearview. The vehicle 100 comprises second and third cameras 102R and 102Lat the positions of side mirrors. The second and third cameras 102R and102L are also capable of capturing the rear view.

As described above, the first to third cameras 101, 102R and 102L areattached to different positions in the vehicle 100. Thus, when eachcamera captures the view behind the vehicle, an effective region whichcan be captured by the field of view (W1) of camera 101 may be a blindregion (an ineffective region or a field of view W3) which cannot becaptured by the field of view of camera 102R or 102L (for example, afield of view W2).

According to the present embodiment of the image synthesis device forthe electronic mirror, the first camera 101 obtains a first image (rearimage) of a first view by the first field of view W1 from a firstposition (for example, the position of the trunk). The second camera102R obtains a second image by the second field of view W2 including aview obstruction (a side of the own vehicle) in the direction of thefirst view from a second position (a lateral position of the driver seator the position of a side door) different from the first position (theposition of the trunk). Thus, the second image includes the image of aside of the own vehicle.

As described later, an image processing device 200 connects a part ofthe first image as a complementary image to a part of the second image.In this case, the image processing device 200 converts the image of theview obstruction (a side of the own vehicle) included in the secondimage into a translucent image, and superimposes the complementary imageon the translucent image. The image processing device 200 obtains athird image by providing a border between the first image and thecomplementary image in the outline of the translucent image (in otherwords, the outline of a side of the own vehicle) and connecting thecomplementary image to the first image.

The complementary image superimposed on the region of the translucentimage is obtained by connecting (or synthesizing or combining) aprojective transformation image based on a far plan position and aprojective transformation image based on a ground position closer to thecapture position than the far plan position.

When the horizontal direction of the first and second cameras is aright-and-left direction, and the perpendicular direction is anupper-and-lower direction, the upper region of the complementary imagecorresponds to an image obtained by applying projective transformationbased on the far plan position, and the lower region of thecomplementary image corresponds to an image obtained by applyingprojective transformation based on the ground position.

The image of the view obstruction undergoes a process for averaging theluminance of a plurality of frames, and is made translucent. In thismanner, a vehicle running side by side, a vehicle coming in the oppositedirection or the light of a lamp is not reflected on the door (having amirror effect) of the own vehicle in the image.

The lower region of the complementary image includes an image filledwith a color similar to the color of the surrounding image. Thus, theportion under the vehicle (for example, a road) is displayed bypresumption. In this manner, the screen can be stable as a whole.

FIG. 3 shows electronic processing blocks for performing the aboveprocesses. FIG. 3 shows the first camera (rear camera) 101, the secondcamera (right side camera) 102R and the third camera (left side camera)102L. The capture signals of cameras 101, 102R and 102L are input to theimage processing device 200.

The capture signal of the first camera 101 is input to an imageextraction module 221 a of a first image processor 221. The capturesignal of the second camera (right side camera) 102R is input to animage extraction module 222 a of a second image processor 222. Thecapture signal of the third camera (left side camera) 102L is input toan image extraction module 223 a of a third image processor 223.

Each of the first to third cameras 101, 102R and 102L obtains an imageof a region of a wide angle including a region wider than the image tobe used. Each of extraction modules 221 a, 222 a and 223 a extracts theimage region to be used from the image of the region of a wide angle.The extraction position and the extraction region may be changed inaccordance with the speed of the vehicle, the vibration of the vehicleand the control direction of the wheel. The normal setting position ofthe extraction position and/or the extraction region may be changed byuser operation. This is also effective when an image is experimentallymonitored or tested.

In the first image processor 221, projective transformation modules 221b and 221 c apply projective transformation to the image signalextracted by image extraction module 221 a. Subsequently, the imagesignal is input to a synthesis module 230. Projective transformationmodules 221 b and 221 c are capable of obtaining a projectivetransformation image based on a far plan position and a projectivetransformation image based on a ground position closer to the captureposition than the far plan position.

Projective transformation module 221 b transforms an image at theviewpoint of the first camera 101 into an image at the viewpoint of thesecond camera 102R and an image at the viewpoint of the third camera102L based on a far plan position. Projective transformation module 221c transforms an image at the viewpoint of the first camera 101 into animage at the viewpoint of the second camera 102R and an image at theviewpoint of the third camera 102L based on a ground position closer tothe capture position than the far plan position.

Thus, an image at the viewpoint of the first camera 101 is input to thesynthesis module 230 as an image at the viewpoint of the second camera102R (in other words, as an image obtained by transformation based on afar plan position and an image obtained by transformation based on aground position). An image at the viewpoint of the first camera 101 isinput to the synthesis module 230 as an image at the viewpoint of thethird camera 102L (in other words, as an image obtained bytransformation based on a far plan position and an image obtained bytransformation based on a ground position).

In the second image processor 222, a projective transformation module222 b applies projective transformation to the image signal extracted byimage extraction module 222 a. This process of projective transformationmodule 222 b may be omitted. Similarly, in the third image processor223, a projective transformation module 223 b applies projectivetransformation to the image signal extracted by image extraction module223 a. This process of projective transformation module 223 b may beomitted. The degree of transformation applied in projectivetransformation modules 222 b and 223 b is less than that of the abovetransformation applied in projective transformation modules 221 b and221 c.

Projective transformation is also called planar projectivetransformation or homography transformation. Projective transformationis a technique for transforming a plane figure in a virtual plane at aviewpoint (a first viewpoint) into a plane figure in a virtual plane atanother viewpoint (a second viewpoint).

The image signal obtained by the second image processor 222 includes animage of a side of the own vehicle (in other words, an image obtained bycapturing a side of the right door in the direction of the rear side; animage of the view obstruction). Similarly, the image signal obtained bythe third image processor 223 includes an image of a side of the ownvehicle (in other words, an image obtained by capturing a side of theleft door in the direction of the rear side; an image of the viewobstruction).

Since the attachment position of the second camera (side camera) 102R inthe vehicle is determined, the position of the region of the image ofthe view obstruction in the second image with respect to the angle ofview is also determined. Similarly, since the attachment position of thethird camera (side camera) 102L in the vehicle is determined, theposition of the region of the image of the view obstruction in the thirdimage with respect to the angle of view is also determined.

A luminance averaging processor 222 c applies a process for averagingthe luminance to the image of the view obstruction (in other words, theimage of the right side of the own vehicle) included in the secondimage. This process is applied such that a vehicle running side by side,a vehicle coming in the opposite direction or the light of a lamp is notreflected on the doors (having a mirror effect) of the own vehicle inthe image. Similarly, a luminance averaging processor 223 c applies aprocess for averaging the luminance to the image of the view obstruction(the image of the left side of the own vehicle) included in the thirdimage.

The image of the view obstruction (the image of the right side of theown vehicle) included in the second image in which the luminance hasbeen averaged is made translucent by a processor for translucence 222 d.The image of the view obstruction (the image of the left side of the ownvehicle) included in the third image in which the luminance has beenaveraged is made translucent by a processor for translucence 223 d. Thedegree of translucence may be arbitrarily controlled by the informationfrom an electronic controller 250.

The second image output from processor for translucence 222 d is inputto an edge processor 222 e such that the image of the view obstructionundergoes an edge (outline) process. Similarly, the third image outputfrom processor for translucence 223 d is input to an edge processor 223e such that the image of the view obstruction undergoes an edge(outline) process.

An edge process is, for example, a process for adding a solid or dashedline (in other words, an edge line) to an outline.

For example, the color, weight and shade of the edge line may be changedin accordance with the driving state of the vehicle or the externalenvironment. These elements may be adjusted by user operation.

For example, when the vehicle is traveling backward, the edge may beclarified such that the distance between an obstruction and a side ofthe vehicle or the traveling direction of the vehicle can be easilynoticed. When the vehicle is making a right turn or a left turn, theedge may be clarified such that the distance between an obstruction anda side of the vehicle or the traveling direction of the vehicle can beeasily noticed.

A first image signal obtained by the above process in the first imageprocessor 221, a second image signal obtained by the above process inthe second image processor 222 and a third image signal obtained by theabove process in the third image processor 223 are synthesized in thesynthesis module 230.

The adjusting and controlling processes described above may be performedbased on a control signal from the electronic controller 250. Theelectronic controller 250 has the information of the speed of thevehicle J1, the information of various types of manual operations J2,the direction information indicating the state of turning right, turningleft, moving straight ahead or moving backward J3, time information J4,lighting information, the information of outside air temperature, theinformation of indoor temperature and other types of information.

The speed information J1 can be used as, for example, adjustmentinformation to automatically adjust the distance between each camera andthe reference plane (reference image) for changing the position of theviewpoint of the camera in projective transformation modules 221 b, 221c, 222 b and 223 b.

The manual operation information J2 includes, for example, informationused to adjust the reference position for extracting a region from animage, and information used to initially set or adjust the referenceplane for projective transformation.

The direction information J3 can be used as control information for aprocess for expanding the region to be extracted from an image, anenlarging process, a process for changing the position of the region tobe extracted, a process for making an image translucent, a process forchanging the weight, color and shade of the edge, etc. For example, whenthe vehicle is moving forward, a process for making an image translucentis applied such that the transparency is made high (in other words, theimage is made highly transparent) by the direction information J3. Whenthe vehicle is making a left turn or a right turn, the edge is made wideand dark by the direction information J3. In this way, the distancebetween the vehicle and an obstruction lateral to the vehicle can beeasily recognized by the driver.

The time information J4 can be used to adjust the sensitivity of eachcamera. When it is dark, the sensitivity of the capture signal of eachcamera may be made high by the time information J4. When it is light,the sensitivity may be made low by the time information J4.

Sensor information includes various types of information such as theinformation of external humidity and the information of outside airtemperature. An appropriate image can be obtained from each camera byusing the sensor information. For example, when it rains outside, acompressed air cleaner provided on a side of each camera may remove themoisture or dust around the lens. An opening for spraying compressed airmay be provided in front of the lens of each camera. A heater may befurther provided in front of the lens of each camera since snow or icemay be attached to the front of the camera.

In this system, a display 226 is provided at a position which is easilyviewed by the driver inside the vehicle. Other displays may beadditionally provided in front of the backseat and the passenger seatsuch that passengers other than the driver can view the displays.

One, some or all of the internal modules of the image processing device200 shown in FIG. 3 may be realized by software, or may be a processingdevice operated by software stored in a storage medium. The software maybe transmitted from outside via a communication module. One, some or allof the internal modules of the image processing device 200 may be adevice which receives and executes the software. A recording/reproducingdevice may be connected to the image processing device 200. For example,an image signal may be recorded in the recording/reproducing device inresponse to a traveling speed, vibration or sound set in particular.

FIG. 4 is an explanatory diagram shown for explaining the basic processfor synthesizing images. A rear image 410 is a rear image extracted froman image captured by the rear camera 101. A side image 420 is a sideimage extracted from an image captured by the right side camera 102R.

FIG. 4 shows the rear image 410 and the side image 420 obtained from theright side camera 102R. However, the actual device also uses a sideimage obtained from the left side camera 102L. Here, to simplifyexplanation, the embodiment is described using the relationship betweenthe rear image 410 and the side image 420.

The side image 420 includes the image (A0) of the side of the vehicle(in other words, the image of the side of the own vehicle or the imageof the view obstruction) (on the left part of the side image 420). Thisobstructive region (the region of image A0) is a blind region in therear view of the vehicle for the right side camera 102R.

The region of image A0 is made translucent (by processor fortranslucence 222 d). A part of the rear image 410 (in other words, theimage of a region corresponding to image A0) is superimposed on thetranslucent image. In this way, the images are synthesized in thesynthesis module 230.

As shown in the synthesized image 430, a part of the rear image issuperimposed on the translucent image of the view obstruction. In thesynthesized image 430, an edge line 435 indicates the border (in otherwords, the combined position, connected position or outline) between thetranslucent image A0 and the image captured by the right side camera102R. The edge line 435 is added by edge processor 222 e.

The edge line 435 is useful for the driver to confirm the width of theown vehicle when the driver moves the vehicle forward or backing thevehicle. The width, color and brightness (shade), etc., of the edge line435 may be changed in accordance with the surrounding state, forexample, depending on whether the outside is dark or light. Thisstructure can raise the security awareness of the driver.

In the synthesized image 430, a non-display line 432 is indicatedbetween the upper region A1 and the lower region A2 of the translucentimage A0. The line 432 is not displayed in the actual image. The line isadded for explanatory purpose.

A large number of diamonds are shown in FIG. 4 and after drawings.However, the diamonds are not displayed in the actual image. Thediamonds are shown to indicate that rectangles have been transformedinto diamonds as a result of transformation of a viewpoint. The diamondsare not displayed in the actual image.

Each image of the upper and lower regions A1 and A2 is an image obtainedby transforming the rear image 410 of the viewpoint of the rear camera101 into the image of the viewpoint of the right side camera 102R. Inthis case, the reference position of transformation of the upper regionA1 is a far plan position 412. The reference position of transformationof the lower region A2 is a ground position 413 closer to the positionof the camera than the far plan position 412. The far plan position 412and the ground position 413 correspond to positions 422 and 423 shown inthe side image 420 of the right side camera 102R.

In FIG. 4, the image of the side of the own vehicle or the image of theview obstruction (in other words, the image of the obstructive region)is an image captured by the right side camera 102R of the vehicle.However, on the actual display 226 (FIG. 3), an image captured by theleft side camera 102L of the vehicle is also displayed.

FIG. 5 shows a first projective transformation image 410H obtained byapplying projective transformation such that the rear image 410 of theviewpoint of the rear camera 101 is transformed into the image of theviewpoint of the right side camera 102R based on the far plan position(the projective transformation is performed by projective transformationmodule 221 b in FIG. 3). FIG. 5 also shows a second projectivetransformation image 420H obtained by applying projective transformationsuch that the rear image 410 of the viewpoint of the rear camera 101 istransformed into the image of the viewpoint of the right side camera102R based on the ground plan position (the projective transformation isperformed by projective transformation module 221 c in FIG. 3).

The far plan position and the ground plan position may be manuallyadjusted by the driver before driving the vehicle. As described later,the positions may be automatically changed to predetermined positions inaccordance with the speed of the vehicle.

A part of the first projective transformation image 410H (in otherwords, the vicinity of P1 including P1) and a part of the secondprojective transformation image 420H (in other words, the vicinity of P2including P2) are extracted and synthesized in the upper and lowerregions A1 and A2 of the translucent image A0, respectively. When theupper and lower regions A1 and A2 are extracted from the first andsecond projective transformation images 410H and 420H, respectively, theupper and lower regions A1 and A2 are extracted such that the image ofeach region is naturally connected to the side image of the secondcamera.

The region of the view obstruction in the side image 420 of the rightside camera 102R includes a blind region for the rear camera 101. Thus,the region of the view obstruction includes the blind regions of both ofthe cameras.

This blind region is an absolute blind region 433. The absolute blindregion 433 corresponds to the region under the rear part of the vehicle(normally, the surface of a road). In FIG. 5, the absolute blind region433 is shown in the lower left part of the synthesized image 430. Sincethe absolute blind region 433 does not have an image, the absolute blindregion 433 may be completely black or white when no process is applied.

In this system, for example, this region is complemented with the imagedata of the color of the image around the absolute blind region 433 or asimilar color.

This system uses all of the images captured by the rear camera 101, theright side camera 102R and the left side camera 102L as effectively aspossible. As a result, the absolute blind region 433 is present.However, the absolute blind region 433 is complemented with the imagedata of the color of the surrounding image or a similar color.

FIG. 6 is a conceptual diagram shown for explaining the transformationof a viewpoint. For example, a viewpoint C1 is assumed to be theviewpoint of the first camera. A viewpoint C2 is assumed to be theviewpoint of the second camera. Pixels A1 to D1 of a reference plane P1of viewpoint C1 are replaced with pixels A to D of a virtual plane P.Subsequently, pixels A to D of virtual plane P are transformed intopixels A2 to D2 of a virtual plane P2 of the second camera of viewpointC2.

When virtual planes P1 and P2 are at the same position (the groundposition), the image of the ground captured by the first camera may betransformed so as to be an image captured by the second camera.

When four points A to D are present in plane P, these points correspondto A1 to D1 in virtual plane P1 in the image captured from viewpoint C1,and correspond to A2 to D2 in virtual plane P2 in the image capturedfrom viewpoint C2.

When the correspondence relationship between A1 to D1 and A2 to D2 isdetermined, the image captured by the first camera can be transformed soas to be the image captured by the second camera. This correspondencerelationship is called projective transformation (or homographytransformation), and can be expressed by a matrix.

In FIG. 7, the upper formula represents the above transformation matrixH.

When the coordinates of A1 to D1 in virtual plane P1 are (xi, yi, 1),the coordinates are transformed into the coordinates (ui, vi, 1) invirtual plane P2 by the transformation matrix H, where i represents 0,1, 2 and 3. The coordinates are expressed by a three-dimensionalhomogeneous coordinate system in which one element is added to atwo-dimensional plane.

The coordinates of A1 to D1 correspond to those of A2 to D2 as follows.

-   -   A1: (x0, y0, 1), B1: (x1, y1, 1), C1: (x2, y2, 1),    -   D1: (x3, y3, 1)    -   A2: (u0, v0, 1), B2: (u1, v1, 1), C2: (u2, v2, 1)    -   D2: (u3, v3, 1)

When (xi, yi, 1) and (ui, vi, 1) are known, matrix elements h0 to h7 canbe calculated.

Since the number of elements of the transformation matrix H is eight,elements h0 to h7 can be calculated when the correspondencerelationships of four points are obtained.

FIG. 7 shows a determinant in the lower part. The determinant isobtained by developing the upper formula for the case of i=0, 1, 2, 3and factoring out h0 to h7. By this determinant, h0 to h7 can becalculated.

By the transformation matrix H, an arbitrary point in virtual plane P1from viewpoint C1 can be transformed into a point in virtual plane P2from viewpoint C2.

FIG. 8 shows an example of an image in which the luminance is averagedby luminance averaging modules 222 c and 223 c shown in FIG. 3. It isassumed that a bus passes by the right side of the own vehicle as shownin an image 510. In this case, the side of the own vehicle functions asa mirror. The bus is reflected on the side of the own vehicle. When thisstate is captured by the right side camera 102R without any process, theimage captured by the right side camera 102R includes the image (B1) ofthe bus (see images 510 and 511).

Since the vehicle is moving, various objects, such as a bus, a landscapeand a house, are reflected on the vehicle. In this system, an imageobtained by averaging the luminance of image B1 of the side of thevehicle is an image B12 (see an image 512). When image B12 obtained byaveraging the luminance is made translucent, the translucent image isnot heavily affected by the image of reflection as shown in an image513. Image 511 is an image obtained when the luminance is not averaged.Thus, image 511 includes an obstructive image in the region of thetranslucent image. However, in the system of the present embodiment, theimage of the translucent region can be easily viewed.

FIG. 9 is an explanatory diagram showing another embodiment when theabsolute blind region 433 is filled. When a fourth camera is attached tothe vehicle to capture the front side, a part of the synthesized image430 can be filled with a partial image 466 of the image captured by thefourth camera. In this case, the system uses an extraction module whichextracts the image captured by the front camera, a transformation modulewhich transforms the front-back direction of the extracted image, and atime adjustment module. The partial image 466 in which the time has beenadjusted is transmitted to the synthesis module 230 as a pseudo image,and is used as the image of the absolute blind region 433.

FIG. 10 shows an example in which the synthesized image 430 is displayedon the display 266 in an electronic mirror using standard cameras. FIG.10 shows the image of the right side. However, the display 226 alsodisplays the image of the left side. In the image of the left side,similarly, a complementary image is superimposed on a translucent image.In the outline of the translucent image, a line indicating the borderbetween the image obtained by the rear camera and the complementaryimage is displayed.

FIG. 11 shows an example of a synthesized image 430A obtained when usingthe first, second and third cameras having an angle of view broader thanthat of a normal camera and configured to capture panoramic images. Thesynthesized image 430A also includes an absolute blind region 431A. Thisregion is filled with a color or pattern similar to the color or patternof the surrounding region.

FIG. 12 is a plan view showing vehicles 502 and 503 moving on a road.The distance from vehicle 502 to the reference plan position (forexample, the far plan position or the ground position shown in FIG. 4)may be automatically adjusted in accordance with the speed. When thespeed is slow (for example, 0 to 35 km/h), the driver may want to payattention to a vehicle which follows the own vehicle and iscomparatively close to the own vehicle (for example, vehicle 503). Whenthe speed is medium (for example, 35 to 65 km/h), the driver may want topay attention to a more distant vehicle which follows the own vehicle.When the speed is high (for example, 65 km/h or higher), the driver maywant to pay attention to a far more distant vehicle which follows theown vehicle.

In this case, the distance from the own vehicle to the reference planposition (for example, the far plan position or the ground positionshown in FIG. 4) may be automatically adjusted by user setting.

FIG. 13 shows an example of the synthesized image 430 when the referenceplan position (virtual plan position) is distant. FIG. 14 shows anexample of the synthesized image 430 when the reference plan position isclose. In the example of FIG. 13, when the reference plan position isdistant, the vertical lines of the distant buildings behind the ownvehicle in the first image conform to those in the second image.However, a misalignment is caused in the portion connecting the firstimage and the second image in the image of the close vehicle.

As shown in FIG. 14, when the reference plan position (virtual planposition) is close, the first image is misaligned with the second imagewith respect to the vertical lines of the distant buildings behind theown vehicle. However, in the image of the close vehicle, the first imageis smoothly connected to the second image.

In terms of the distance, an appropriate object present behind the ownvehicle can be clarified by adjusting the virtual plan position inaccordance with the speed of the vehicle.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An image synthesis device for an electronicmirror, the device comprising: a rear camera which obtains a first imageof a first view from a first position of a vehicle, as a rear view ofthe vehicle, by a first field of view; a side camera which obtains asecond image from a second position on a more front side than the firstposition, by a second field of view including a view obstruction as aside part of the vehicle, in a direction of the first view; atranslucent image processor which converts an image of the viewobstruction included in the second image into a translucent image; animage synthesizer which obtains a third image by synthesizing aprojective transformation image, as a part of the first image, subjectedto projective transformation into a view position of the side camera andcorresponding to a part of the translucent image of the second image,wherein the projective transformation image includes a first projectivetransformation image obtained by subjecting an image of a viewpoint ofthe rear camera to projective transformation into an image of a viewposition of the side camera, based on a far plan position on the rearside of the vehicle, by a first image processor, and a second projectivetransformation image obtained by subjecting an image of a viewpoint ofthe rear camera to projective transformation into an image of aviewpoint of the side camera based on a ground position closer to acapture position of the side camera than the far plan position, by asecond image processor; and in the first projective transformation imageor the second projective transformation image, the far plan position orthe ground position is adjusted by a setting made in accordance with aspeed of the vehicle.
 2. The image synthesis device of claim 1, furthercomprising: an edge processor capable of adding a line to an outline ofthe translucent image in the second image.
 3. The image synthesis deviceof claim 1, wherein when a direction of a horizontal line of the rearand side cameras is a right-and-left direction, and a direction of aperpendicular line is an upper-and-lower direction, the first imageprocessor generates the first projective transformation image bysubjecting an upper region of the projective transformation image toprojective transformation based on the far plan position, and the secondimage processor generates the second projective transformation image bysubjecting a lower region of the projective transformation image toprojective transformation based on the ground position.
 4. The imagesynthesis device of claim 1, further comprising: a unit which changesany one of a thickness, a color, and concentration of the line.
 5. Theimage synthesis device of claim 1, wherein the synthesizer generates animage filled with a color similar to a color of a surrounding image, asan image to be further added to the projective transformation image. 6.An image synthesis method of an electronic mirror, the method using: inan electronic mirror mounted on a vehicle to display an image of a rearside of the vehicle, a rear camera which obtains a first image of afirst view from a first position, as a rear view of the vehicle, by afirst field of view; a side camera which obtains a second image from asecond position on a more front side than the first position, by asecond field of view including a view obstruction as a side part of thevehicle, in a direction of the first view; and an image processor whichproceeds the first image and the second image from the rear and sidecameras, the method comprising: converting an image of the viewobstruction included in the second image into a translucent image;adding a line to an outline of the translucent image in the secondimage; and obtaining a third image by synthesizing a projectivetransformation image, as a part of the first image, subjected toprojective transformation into a view position of the side camera andcorresponding to a part of the translucent image of the second image,wherein when the projective transformation image is generated, a firstprojective transformation image obtained by subjecting an image of aviewpoint of the rear camera to projective transformation into an imageof a view position of the side camera, based on a far plan position onthe rear side of the vehicle, is generated by a first image processor, asecond projective transformation image obtained by subjecting an imageof a viewpoint of the side camera to projective transformation into animage of a viewpoint of the rear camera based on a ground positioncloser to a capture position of the side camera than the far planposition, is generated by a second image processor, in the firstprojective transformation image or the second projective transformationimage, the far plan position or the ground position is adjusted by asetting made in accordance with a speed of the vehicle, and theprojective transformation image is generated by connecting the firstprojective transformation image and the second projective transformationimage.
 7. The method of claim 6, further comprising: adding a line to anoutline of the translucent image in the second image.
 8. The method ofclaim 6 wherein when a direction of a horizontal line of the rear andside cameras is a right-and-left direction, and a direction of aperpendicular line is an upper-and-lower direction, the first projectivetransformation image is generated by subjecting an upper region of theprojective transformation image to projective transformation based onthe far plan position, and the second projective transformation image isgenerated by subjecting a lower region of the projective transformationimage to projective transformation based on the ground position.
 9. Themethod of claim 6, further comprising: changing any one of a thickness,a color, and concentration of the line.
 10. The method of claim 6,wherein an image filled with a color similar to a color of a surroundingimage, is generated as an image to be further added to the projectivetransformation image.